Dual Space Preconditioning for Gradient Descent

نویسندگان

چکیده

The conditions of relative smoothness and strong convexity were recently introduced for the analysis Bregman gradient methods convex optimization. We introduce a generalized left-pr...

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Dual Space Gradient Descent for Online Learning

One crucial goal in kernel online learning is to bound the model size. Common approaches employ budget maintenance procedures to restrict the model sizes using removal, projection, or merging strategies. Although projection and merging, in the literature, are known to be the most effective strategies, they demand extensive computation whilst removal strategy fails to retain information of the r...

متن کامل

Nonsymmetric Preconditioning for Conjugate Gradient and Steepest Descent Methods1

We numerically analyze the possibility of turning off postsmoothing (relaxation) in geometric multigrid when used as a preconditioner in conjugate gradient linear and eigenvalue solvers for the 3D Laplacian. The geometric Semicoarsening Multigrid (SMG) method is provided by the hypre parallel software package. We solve linear systems using two variants (standard and flexible) of the preconditio...

متن کامل

Steepest Descent and Conjugate Gradient Methods with Variable Preconditioning

We analyze the conjugate gradient (CG) method with variable preconditioning for solving a linear system with a real symmetric positive definite (SPD) matrix of coefficients A. We assume that the preconditioner is SPD on each step, and that the condition number of the preconditioned system matrix is bounded above by a constant independent of the step number. We show that the CG method with varia...

متن کامل

L1 Regression using Lewis Weights Preconditioning and Stochastic Gradient Descent

We consider the l1 minimization problem minx ‖Ax − b‖1 in the overconstrained case, commonly known as the Least Absolute Deviations problem, where there are far more constraints than variables. More specifically, we have A ∈ R for n ≫ d. Many important combinatorial problems, such as minimum cut and shortest path, can be formulated as l1 regression problems [CMMP13]. We follow the general parad...

متن کامل

Learning by Gradient Descent in Function Space

Traditional connectionist networks have homogeneous nodes wherein each node executes the same function. Networks where each node executes a di erent function can be used to achieve e cient supervised learning. A modi ed back-propagation algorithm for such networks, which performs gradient descent in \function space," is presented and its advantages are discussed. The bene ts of the suggested pa...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Siam Journal on Optimization

سال: 2021

ISSN: ['1095-7189', '1052-6234']

DOI: https://doi.org/10.1137/19m130858x